Goto

Collaborating Authors

 facial feature


CSGaze: Context-aware Social Gaze Prediction

Madan, Surbhi, Ghosh, Shreya, Subramanian, Ramanathan, Dhall, Abhinav, Gedeon, Tom

arXiv.org Artificial Intelligence

A person's gaze offers valuable insights into their focus of attention, level of social engagement, and confidence. In this work, we investigate how contextual cues combined with visual scene and facial information can be effectively utilized to predict and interpret social gaze patterns during conversational interactions. We introduce CSGaze, a context aware multimodal approach that leverages facial, scene information as complementary inputs to enhance social gaze pattern prediction from multi-person images. The model also incorporates a fine-grained attention mechanism centered on the principal speaker, which helps in better modeling social gaze dynamics. Experimental results show that CSGaze performs competitively with state-of-the-art methods on GP-Static, UCO-LAEO and AVA-LAEO. Our findings highlight the role of contextual cues in improving social gaze prediction. Additionally, we provide initial explainability through generated attention scores, offering insights into the model's decision-making process. We also demonstrate our model's generalizability by testing our model on open set datasets that demonstrating its robustness across diverse scenarios.


Relative Uncertainty Learning for Facial Expression Recognition

Neural Information Processing Systems

In facial expression recognition (FER), the uncertainties introduced by inherent noises like ambiguous facial expressions and inconsistent labels raise concerns about the credibility of recognition results. To quantify these uncertainties and achieve good performance under noisy data, we regard uncertainty as a relative concept and propose an innovative uncertainty learning method called Relative Uncertainty Learning (RUL). Rather than assuming Gaussian uncertainty distributions for all datasets, RUL builds an extra branch to learn uncertainty from the relative difficulty of samples by feature mixup. Specifically, we use uncertainties as weights to mix facial features and design an add-up loss to encourage uncertainty learning. It is easy to implement and adds little or no extra computation overhead.


Cross-Enhanced Multimodal Fusion of Eye-Tracking and Facial Features for Alzheimer's Disease Diagnosis

Nie, Yujie, Ni, Jianzhang, Ye, Yonglong, Zhang, Yuan-Ting, Wing, Yun Kwok, Xu, Xiangqing, Ma, Xin, Fan, Lizhou

arXiv.org Artificial Intelligence

Accurate diagnosis of Alzheimer's disease (AD) is essential for enabling timely intervention and slowing disease progression. Multimodal diagnostic approaches offer considerable promise by integrating complementary information across behavioral and perceptual domains. Eye-tracking and facial features, in particular, are important indicators of cognitive function, reflecting attentional distribution and neurocognitive state. However, few studies have explored their joint integration for auxiliary AD diagnosis. In this study, we propose a multimodal cross-enhanced fusion framework that synergistically leverages eye-tracking and facial features for AD detection. The framework incorporates two key modules: (a) a Cross-Enhanced Fusion Attention Module (CEF AM), which models inter-modal interactions through cross-attention and global enhancement, and (b) a Direction-Aware Convolution Module (DACM), which captures fine-grained directional facial features via horizontal-vertical receptive fields. To support this work, we constructed a synchronized multimodal dataset, including 25 patients with AD and 25 healthy controls (HC), by recording aligned facial video and eye-tracking sequences during a visual memory-search paradigm, providing an ecologically valid resource for evaluating integration strategies. Extensive experiments on this dataset demonstrate that our framework outperforms traditional late fusion and feature concatenation methods, achieving a classification accuracy of 95.11% in distinguishing AD from HC, highlighting superior robustness and diagnostic performance by explicitly modeling inter-modal dependencies and modality-specific contributions. Introduction Alzheimer's disease (AD), a progressive and irreversible neurodegenera-tive disorder, represents the primary cause of dementia in older adults [1]. It typically begins with mild memory loss and gradually progresses to severe impairments in executive and cognitive functions [2]. Within the global aging population, more than 150 million people worldwide will be affected by AD or other forms of dementia [3], imposing a substantial burden on both families and healthcare systems. Early and accurate identification of Alzheimer's disease is vital to initiate interventions that may slow progression and improve quality of life. Clinically, the diagnosis of AD primarily relies on biomarker analysis, neu-roimaging techniques, and neuropsychological assessments.


When Face Recognition Doesn't Know Your Face Is a Face

WIRED

When Face Recognition Doesn't Know Your Face Is a Face An estimated 100 million people live with facial differences. As face recognition tech becomes widespread, some say they're getting blocked from accessing essential systems and services. Autumn Gardiner thought updating her driving license would be straightforward. After getting married last year, she headed to the local Department of Motor Vehicles office in Connecticut to get her name changed on her license. While she was there, Gardiner recalls, officials said she needed to update her photo.


The Influence of Facial Features on the Perceived Trustworthiness of a Social Robot

Barrow, Benedict, Moore, Roger K.

arXiv.org Artificial Intelligence

Abstract-- Trust and the perception of trustworthiness play an important role in decision-making and our behaviour towards others, and this is true not only of human-human interactions but also of human-robot interactions. While significant advances have been made in recent years in the field of social robotics, there is still some way to go before we fully understand the factors that influence human trust in robots. This paper presents the results of a study into the first impressions created by a social robot's facial features, based on the hypothesis that a'babyface' engenders trust. By manipulating the back-projected face of a Furhat robot, the study confirms that eye shape and size have a significant impact on the perception of trustworthiness. The work thus contributes to an understanding of the design choices that need to be made when developing social robots so as to optimise the effectiveness of human-robot interaction. Trust is a fundamental building block for any society to function properly.



The Importance of Facial Features in Vision-based Sign Language Recognition: Eyes, Mouth or Full Face?

Pham, Dinh Nam, Avramidis, Eleftherios

arXiv.org Artificial Intelligence

Non-manual facial features play a crucial role in sign language communication, yet their importance in automatic sign language recognition (ASLR) remains underexplored. While prior studies have shown that incorporating facial features can improve recognition, related work often relies on hand-crafted feature extraction and fails to go beyond the comparison of manual features versus the combination of manual and facial features. In this work, we systematically investigate the contribution of distinct facial regionseyes, mouth, and full faceusing two different deep learning models (a CNN-based model and a transformer-based model) trained on an SLR dataset of isolated signs with randomly selected classes. Through quantitative performance and qualitative saliency map evaluation, we reveal that the mouth is the most important non-manual facial feature, significantly improving accuracy. Our findings highlight the necessity of incorporating facial features in ASLR.


Denmark Seeks to Give People Copyright to Their Own Features in Effort to Combat AI Deepfakes

TIME - Tech

The Danish government revealed Thursday that a broad coalition of legislators are working on a bill that would make deepfakes illegal to share and put legal protections in place to prevent AI material depicting a person from being disseminated without their consent. "In the bill we agree and are sending an unequivocal message that everybody has the right to their own body, their own voice and their own facial features, which is apparently not how the current law is protecting people against generative AI," Danish culture minister, Jakob Engel-Schmidt, told The Guardian. The Danish department of culture will submit a proposed amendment for consultation this summer. The bill, if enacted, would issue "severe fines" for online platforms that do not abide by the new law. The Danish government said that parodies and satire would not be affected by the proposed amendment.


Facial Foundational Model Advances Early Warning of Coronary Artery Disease from Live Videos with DigitalShadow

Zhou, Juexiao, Han, Zhongyi, Xin, Mankun, He, Xingwei, Wang, Guotao, Song, Jiaoyan, Luo, Gongning, He, Wenjia, Li, Xintong, Chu, Yuetan, Chen, Juanwen, Wang, Bo, Wu, Xia, Duan, Wenwen, Guo, Zhixia, Bai, Liyan, Pan, Yilin, Bi, Xuefei, Liu, Lu, Feng, Long, He, Xiaonan, Gao, Xin

arXiv.org Artificial Intelligence

Abstract--Global population aging presents increasing challenges to healthcare systems, with coronary artery disease (CAD) responsible for approximately 17.8 million deaths annually, making it a leading cause of global mortality . As CAD is largely preventable, early detection and proactive management are essential. In this work, we introduce DigitalShadow, an advanced early warning system for CAD, powered by a fine-tuned facial foundation model. The system is pre-trained on 21 million facial images and subsequently fine-tuned into LiveCAD, a specialized CAD risk assessment model trained on 7,004 facial images from 1,751 subjects across four hospitals in China. DigitalShadow functions passively and contactlessly, extracting facial features from live video streams without requiring active user engagement. Integrated with a personalized database, it generates natural language risk reports and individualized health recommendations. With privacy as a core design principle, DigitalShadow supports local deployment to ensure secure handling of user data. The world's population is rapidly ageing [1], with significant implications for the prevalence of chronic diseases such as Coronary Artery Disease (CAD) [2], affecting not only individuals but also families and societies at large [3]. The number of older people is increasing at an unprecedented rate, projected to grow from approximately 761 million in 2021 to 1.6 billion by 2050, which would represent nearly 16% of the global population, according to the UN's W orld Social Report 2023 [4]. The aging population presents numerous challenges, including increased pressure on healthcare systems, pension schemes, and long-term care facilities, alongside potential economic consequences, which together fuel growing demand for healthcare services [5], [6]. With advancing age, people become more vulnerable to various critical diseases [7], such as CAD [8], stroke [9], cancer [10], and Parkinson's disease (PD) [11], [12], [13], leading to considerable morbidity and mortality [14].


The facial feature that means you're more likely to have a son

Daily Mail - Science & tech

You might think that having a boy or a girl is completely up to chance. But expectant parents might be able to hazard a good guess – depending on what the father's facial features are like. Researchers wanted to find out whether certain traits in parents were linked to the sex of their firstborn. The team, from the University of Michigan, recruited 104 pairs of parents with at least one child. Both were asked to submit facial photographs which were rated for attractiveness, dominance and masculinity or femininity by university students.